Table of Contents


Open question WHYQUALITY (answered by all respondents by typing in)

InterRater Reliability (IRR)

Cohen’s Kappa and percentage of agreement across Coders 1 and 2 for each variable studied
Metric Cohen_Kappa Agreement_Percentage
VALID 0.68 91.44
ABBR 0.46 98.06
PB 0.19 97.94
NTHEMES 0.62 73.61

Valid answers

Proportions per group

Proportions of non sense, not in line, DK and valid answers using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
1 Non sense 0.4 2.1 0.8 0.8
10 Depends 5.7 7.4 9.6 6.1
2 Not in line 4.5 3.7 3.2 6.1
3 DK 5.7 3.7 4.0 4.9
4 Valid 83.6 83.1 82.3 82.0
Number of observations 244 243 249 245
Proportions of non sense, not in line, DK and valid answers using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
1 Non sense 0.4 1.2 0.8 0.8
10 Depends 2.5 7.0 6.0 6.1
2 Not in line NA 1.6 NA 2.9
3 DK 6.6 2.9 4.8 5.3
4 Valid 90.6 87.2 88.0 84.9
9 To be defined NA NA 0.4 NA
Number of observations 244 243 249 245
Proportions of non sense, not in line, DK and valid answers using the codes of Coder 3

Coder 3 Control Choice PushDictation PushRecording
1 Non sense 0.4 2.1 0.8 0.8
10 Depends 6.1 7.8 8.4 5.7
2 Not in line 2.5 2.5 2.4 5.7
3 DK 5.7 3.7 3.6 4.9
4 Valid 85.2 84.0 84.7 82.9
Number of observations 244 243 249 245

Tests of significance across groups

Tests non sense Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 1.000 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.216 ns 1 ns
PushDictation Choice 0.427 ns 1 ns
PushRecording Choice 0.440 ns 1 ns
Tests non sense Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 1.000 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.613 ns 1 ns
PushDictation Choice 0.978 ns 1 ns
PushRecording Choice 0.993 ns 1 ns
Tests non sense Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1 ns
Control PushRecording 1.000 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.216 ns 1 ns
PushDictation Choice 0.427 ns 1 ns
PushRecording Choice 0.440 ns 1 ns
Tests not in line Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.608 ns 1 ns
Control PushRecording 0.553 ns 1 ns
PushDictation PushRecording 0.186 ns 1 ns
Control Choice 0.827 ns 1 ns
PushDictation Choice 0.959 ns 1 ns
PushRecording Choice 0.305 ns 1 ns
Tests not in line Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushRecording 0.0227
0.106 ns
PushDictation PushRecording 0.0211
0.106 ns
Control Choice 0.1310 ns 0.378 ns
PushDictation Choice 0.1260 ns 0.378 ns
PushRecording Choice 0.5510 ns 0.551 ns
Tests not in line Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1.000 ns
Control PushRecording 0.112 ns 0.612 ns
PushDictation PushRecording 0.102 ns 0.612 ns
Control Choice 1.000 ns 1.000 ns
PushDictation Choice 1.000 ns 1.000 ns
PushRecording Choice 0.114 ns 0.612 ns
Tests DK Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.497 ns 1 ns
Control PushRecording 0.832 ns 1 ns
PushDictation PushRecording 0.797 ns 1 ns
Control Choice 0.398 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.669 ns 1 ns
Tests DK Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.5230 ns 1.000 ns
Control PushRecording 0.6930 ns 1.000 ns
PushDictation PushRecording 0.9670 ns 1.000 ns
Control Choice 0.0893 ns 0.536 ns
PushDictation Choice 0.3780 ns 1.000 ns
PushRecording Choice 0.2610 ns 1.000 ns
Tests DK Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.366 ns 1 ns
Control PushRecording 0.832 ns 1 ns
PushDictation PushRecording 0.628 ns 1 ns
Control Choice 0.398 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.669 ns 1 ns
Tests valid Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.797 ns 1 ns
Control PushRecording 0.735 ns 1 ns
PushDictation PushRecording 1.000 ns 1 ns
Control Choice 0.984 ns 1 ns
PushDictation Choice 0.909 ns 1 ns
PushRecording Choice 0.844 ns 1 ns
Tests valid Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.4270 ns 1.000 ns
Control PushRecording 0.0759 ns 0.455 ns
PushDictation PushRecording 0.3900 ns 1.000 ns
Control Choice 0.3050 ns 1.000 ns
PushDictation Choice 0.9190 ns 1.000 ns
PushRecording Choice 0.5370 ns 1.000 ns
Tests valid Coder 3
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.975 ns 1 ns
Control PushRecording 0.550 ns 1 ns
PushDictation PushRecording 0.656 ns 1 ns
Control Choice 0.787 ns 1 ns
PushDictation Choice 0.907 ns 1 ns
PushRecording Choice 0.839 ns 1 ns



For valid answers only from now on


Abbreviations

Proportions per group

Abbreviations, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Abbreviation(s) 3.4 1.5 1.4 2.0
None 96.6 98.5 98.6 98.0
Number of observations 208 204 211 203
Abbreviations, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Abbreviation(s) 1.4 0.5 1.4 3.0
None 98.6 99.5 98.6 97.0
Number of observations 208 204 211 203

Tests of significance across groups

Tests abbreviations Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.326 ns 1 ns
Control PushRecording 0.568 ns 1 ns
PushDictation PushRecording 0.959 ns 1 ns
Control Choice 0.353 ns 1 ns
PushDictation Choice 1.000 ns 1 ns
PushRecording Choice 0.995 ns 1 ns
Tests abbreviations Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1.000 ns 1.000 ns
Control PushRecording 0.477 ns 1.000 ns
PushDictation PushRecording 0.464 ns 1.000 ns
Control Choice 0.629 ns 1.000 ns
PushDictation Choice 0.639 ns 1.000 ns
PushRecording Choice 0.126 ns 0.754 ns


Problems detected

Proportions per group

Problems, using the codes of Coder 1

Coder 1 Control Choice PushDictation PushRecording
Incomplete sentence(s) 1.0 1.0 2.4 NA
No problem detected 97.6 98.5 95.3 100.0
Other problem(s) 1.4 0.5 2.4 NA
Number of observations 208 204 211 203
Problems, using the codes of Coder 2

Coder 2 Control Choice PushDictation PushRecording
Incomplete sentence(s) 0.5 0.5 0.5 NA
No problem detected 99.5 99.5 99.5 100.0
Number of observations 208 204 211 203

Tests of significance across groups

Tests problems Coder 1: no problem detected
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.30600 ns 0.7450 ns
Control PushRecording 0.07630 ns 0.3810 ns
PushDictation PushRecording 0.00481 ** 0.0288
Control Choice 0.74200 ns 0.7450 ns
PushDictation Choice 0.10300 ns 0.4130 ns
PushRecording Choice 0.24800 ns 0.7450 ns
Tests problems Coder 2: no problem detected
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1 ns 1 ns
Control PushRecording 1 ns 1 ns
PushDictation PushRecording 1 ns 1 ns
Control Choice 1 ns 1 ns
PushDictation Choice 1 ns 1 ns
PushRecording Choice 1 ns 1 ns
Tests problems Coder 1: Incomplete sentence(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.457 ns 1.000 ns
Control PushRecording 0.489 ns 1.000 ns
PushDictation PushRecording 0.079 ns 0.474 ns
Control Choice 1.000 ns 1.000 ns
PushDictation Choice 0.473 ns 1.000 ns
PushRecording Choice 0.481 ns 1.000 ns
Tests problems Coder 2: Incomplete sentence(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 1 ns 1 ns
Control PushRecording 1 ns 1 ns
PushDictation PushRecording 1 ns 1 ns
Control Choice 1 ns 1 ns
PushDictation Choice 1 ns 1 ns
PushRecording Choice 1 ns 1 ns
Tests problems Coder 1: Other problem(s)
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.736 ns 1.000 ns
Control PushRecording 0.255 ns 1.000 ns
PushDictation PushRecording 0.079 ns 0.474 ns
Control Choice 0.629 ns 1.000 ns
PushDictation Choice 0.233 ns 1.000 ns
PushRecording Choice 1.000 ns 1.000 ns
Tests problems Coder 2: Other problem(s)


We do not have any “Other problem(s)” for this coder.



Number of themes

Proportions per group

Number of themes, using the codes of Coder 1: average and median

Coder 1 Control Choice PushDictation PushRecording
mean 1.67 1.58 1.68 1.54
median 1.00 1.00 1.00 1.00
Number of themes, using the codes of Coder 2: average and median

Coder 2 Control Choice PushDictation PushRecording
mean 1.98 1.93 2.09 1.88
median 2.00 2.00 2.00 2.00
Tests differences average number of themes Coder 1
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.894 ns 1 ns
Control PushRecording 0.270 ns 1 ns
PushDictation PushRecording 0.217 ns 1 ns
Control Choice 0.472 ns 1 ns
PushDictation Choice 0.396 ns 1 ns
PushRecording Choice 0.706 ns 1 ns
Tests differences average number of themes Coder 2
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.3090 ns 1.000 ns
Control PushRecording 0.3820 ns 1.000 ns
PushDictation PushRecording 0.0592 ns 0.355 ns
Control Choice 0.6310 ns 1.000 ns
PushDictation Choice 0.1350 ns 0.677 ns
PushRecording Choice 0.6940 ns 1.000 ns


Number of characters

Number of characters: average and median

No. Char Control Choice PushDictation PushRecording
mean 90 88 93 85
median 70 69 72 64
Tests differences average number of characters
group1 group2 p p.signif p.adj p.adj.signif
Control PushDictation 0.787 ns 1 ns
Control PushRecording 0.486 ns 1 ns
PushDictation PushRecording 0.332 ns 1 ns
Control Choice 0.728 ns 1 ns
PushDictation Choice 0.535 ns 1 ns
PushRecording Choice 0.729 ns 1 ns